-
Notifications
You must be signed in to change notification settings - Fork 288
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add debugger benchmarks #7182
Add debugger benchmarks #7182
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 1 performance regressions! Performance is the same for 48 metrics, 14 unstable metrics.
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.35.0-SNAPSHOT~e650df895a, baseline=1.37.0-SNAPSHOT~179ce68ab6
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.07 s) : 0, 1069760
Total [baseline] (10.434 s) : 0, 10433825
Agent [candidate] (1.072 s) : 0, 1072119
Total [candidate] (10.442 s) : 0, 10441859
section appsec
Agent [baseline] (1.186 s) : 0, 1186201
Total [baseline] (10.535 s) : 0, 10534943
Agent [candidate] (1.182 s) : 0, 1181584
Total [candidate] (10.61 s) : 0, 10609828
section iast
Agent [baseline] (1.178 s) : 0, 1178261
Total [baseline] (10.832 s) : 0, 10831571
Agent [candidate] (1.169 s) : 0, 1169480
Total [candidate] (10.739 s) : 0, 10738864
section profiling
Agent [baseline] (1.271 s) : 0, 1271075
Total [baseline] (10.621 s) : 0, 10621315
Agent [candidate] (1.269 s) : 0, 1268998
Total [candidate] (10.652 s) : 0, 10652409
gantt
title petclinic - break down per module: candidate=1.35.0-SNAPSHOT~e650df895a, baseline=1.37.0-SNAPSHOT~179ce68ab6
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (669.969 ms) : 0, 669969
BytebuddyAgent [candidate] (662.948 ms) : 0, 662948
GlobalTracer [baseline] (306.623 ms) : 0, 306623
GlobalTracer [candidate] (315.379 ms) : 0, 315379
AppSec [baseline] (50.089 ms) : 0, 50089
AppSec [candidate] (50.514 ms) : 0, 50514
Remote Config [baseline] (674.416 µs) : 0, 674
Remote Config [candidate] (721.577 µs) : 0, 722
Telemetry [baseline] (7.59 ms) : 0, 7590
Telemetry [candidate] (7.639 ms) : 0, 7639
section appsec
BytebuddyAgent [baseline] (679.192 ms) : 0, 679192
BytebuddyAgent [candidate] (676.631 ms) : 0, 676631
GlobalTracer [baseline] (299.218 ms) : 0, 299218
GlobalTracer [candidate] (297.65 ms) : 0, 297650
AppSec [baseline] (153.924 ms) : 0, 153924
AppSec [candidate] (153.706 ms) : 0, 153706
IAST [baseline] (21.539 ms) : 0, 21539
IAST [candidate] (22.548 ms) : 0, 22548
Remote Config [baseline] (634.763 µs) : 0, 635
Remote Config [candidate] (967.575 µs) : 0, 968
Telemetry [baseline] (7.747 ms) : 0, 7747
Telemetry [candidate] (6.901 ms) : 0, 6901
section iast
BytebuddyAgent [baseline] (786.139 ms) : 0, 786139
BytebuddyAgent [candidate] (780.08 ms) : 0, 780080
GlobalTracer [baseline] (296.542 ms) : 0, 296542
GlobalTracer [candidate] (293.661 ms) : 0, 293661
AppSec [baseline] (47.684 ms) : 0, 47684
AppSec [candidate] (46.821 ms) : 0, 46821
IAST [baseline] (26.68 ms) : 0, 26680
IAST [candidate] (26.587 ms) : 0, 26587
Remote Config [baseline] (577.688 µs) : 0, 578
Remote Config [candidate] (592.202 µs) : 0, 592
Telemetry [baseline] (7.081 ms) : 0, 7081
Telemetry [candidate] (8.485 ms) : 0, 8485
section profiling
BytebuddyAgent [baseline] (667.031 ms) : 0, 667031
BytebuddyAgent [candidate] (668.279 ms) : 0, 668279
GlobalTracer [baseline] (390.728 ms) : 0, 390728
GlobalTracer [candidate] (387.682 ms) : 0, 387682
AppSec [baseline] (51.844 ms) : 0, 51844
AppSec [candidate] (51.113 ms) : 0, 51113
Remote Config [baseline] (667.373 µs) : 0, 667
Remote Config [candidate] (779.031 µs) : 0, 779
Telemetry [baseline] (7.364 ms) : 0, 7364
Telemetry [candidate] (7.374 ms) : 0, 7374
ProfilingAgent [baseline] (96.054 ms) : 0, 96054
ProfilingAgent [candidate] (96.752 ms) : 0, 96752
Profiling [baseline] (96.078 ms) : 0, 96078
Profiling [candidate] (96.776 ms) : 0, 96776
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.35.0-SNAPSHOT~e650df895a, baseline=1.37.0-SNAPSHOT~179ce68ab6
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.076 s) : 0, 1075638
Total [baseline] (8.593 s) : 0, 8593222
Agent [candidate] (1.064 s) : 0, 1063743
Total [candidate] (8.58 s) : 0, 8580271
section iast
Agent [baseline] (1.177 s) : 0, 1177113
Total [baseline] (9.006 s) : 0, 9006158
Agent [candidate] (1.17 s) : 0, 1170070
Total [candidate] (9.018 s) : 0, 9017844
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.186 s) : 0, 1186356
Total [baseline] (9.04 s) : 0, 9039589
Agent [candidate] (1.176 s) : 0, 1176343
Total [candidate] (9.016 s) : 0, 9015919
section iast_TELEMETRY_OFF
Agent [baseline] (1.18 s) : 0, 1179755
Total [baseline] (9.019 s) : 0, 9018660
Agent [candidate] (1.167 s) : 0, 1167344
Total [candidate] (9.019 s) : 0, 9019491
gantt
title insecure-bank - break down per module: candidate=1.35.0-SNAPSHOT~e650df895a, baseline=1.37.0-SNAPSHOT~179ce68ab6
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.025 ms) : 0, 674025
BytebuddyAgent [candidate] (655.505 ms) : 0, 655505
GlobalTracer [baseline] (307.671 ms) : 0, 307671
GlobalTracer [candidate] (314.909 ms) : 0, 314909
AppSec [baseline] (50.621 ms) : 0, 50621
AppSec [candidate] (50.479 ms) : 0, 50479
Remote Config [baseline] (750.41 µs) : 0, 750
Remote Config [candidate] (711.662 µs) : 0, 712
Telemetry [baseline] (7.688 ms) : 0, 7688
Telemetry [candidate] (7.67 ms) : 0, 7670
section iast
BytebuddyAgent [baseline] (784.154 ms) : 0, 784154
BytebuddyAgent [candidate] (780.357 ms) : 0, 780357
GlobalTracer [baseline] (296.443 ms) : 0, 296443
GlobalTracer [candidate] (293.867 ms) : 0, 293867
AppSec [baseline] (47.359 ms) : 0, 47359
AppSec [candidate] (46.486 ms) : 0, 46486
IAST [baseline] (28.129 ms) : 0, 28129
IAST [candidate] (27.87 ms) : 0, 27870
Remote Config [baseline] (561.942 µs) : 0, 562
Remote Config [candidate] (583.079 µs) : 0, 583
Telemetry [baseline] (6.972 ms) : 0, 6972
Telemetry [candidate] (7.613 ms) : 0, 7613
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (790.494 ms) : 0, 790494
BytebuddyAgent [candidate] (784.793 ms) : 0, 784793
GlobalTracer [baseline] (298.495 ms) : 0, 298495
GlobalTracer [candidate] (295.552 ms) : 0, 295552
AppSec [baseline] (47.854 ms) : 0, 47854
AppSec [candidate] (46.885 ms) : 0, 46885
IAST [baseline] (27.423 ms) : 0, 27423
IAST [candidate] (26.702 ms) : 0, 26702
Remote Config [baseline] (600.908 µs) : 0, 601
Remote Config [candidate] (586.646 µs) : 0, 587
Telemetry [baseline] (7.837 ms) : 0, 7837
Telemetry [candidate] (8.366 ms) : 0, 8366
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (785.99 ms) : 0, 785990
BytebuddyAgent [candidate] (779.735 ms) : 0, 779735
GlobalTracer [baseline] (297.331 ms) : 0, 297331
GlobalTracer [candidate] (294.308 ms) : 0, 294308
AppSec [baseline] (47.537 ms) : 0, 47537
AppSec [candidate] (51.326 ms) : 0, 51326
IAST [baseline] (27.64 ms) : 0, 27640
IAST [candidate] (21.16 ms) : 0, 21160
Remote Config [baseline] (631.033 µs) : 0, 631
Remote Config [candidate] (582.117 µs) : 0, 582
Telemetry [baseline] (6.984 ms) : 0, 6984
Telemetry [candidate] (6.87 ms) : 0, 6870
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.35.0-SNAPSHOT~e650df895a, baseline=1.37.0-SNAPSHOT~179ce68ab6
dateFormat X
axisFormat %s
section baseline
no_agent (1.343 ms) : 1325, 1362
. : milestone, 1343,
appsec (1.708 ms) : 1684, 1731
. : milestone, 1708,
appsec_no_iast (1.726 ms) : 1702, 1750
. : milestone, 1726,
iast (1.47 ms) : 1447, 1493
. : milestone, 1470,
profiling (1.487 ms) : 1463, 1512
. : milestone, 1487,
tracing (1.475 ms) : 1450, 1499
. : milestone, 1475,
section candidate
no_agent (1.348 ms) : 1329, 1367
. : milestone, 1348,
appsec (1.735 ms) : 1711, 1759
. : milestone, 1735,
appsec_no_iast (1.692 ms) : 1667, 1717
. : milestone, 1692,
iast (1.486 ms) : 1462, 1510
. : milestone, 1486,
profiling (1.488 ms) : 1463, 1512
. : milestone, 1488,
tracing (1.459 ms) : 1435, 1484
. : milestone, 1459,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.35.0-SNAPSHOT~e650df895a, baseline=1.37.0-SNAPSHOT~179ce68ab6
dateFormat X
axisFormat %s
section baseline
no_agent (367.384 µs) : 348, 387
. : milestone, 367,
iast (474.317 µs) : 453, 496
. : milestone, 474,
iast_FULL (548.218 µs) : 527, 569
. : milestone, 548,
iast_GLOBAL (509.186 µs) : 487, 531
. : milestone, 509,
iast_HARDCODED_SECRET_DISABLED (476.485 µs) : 456, 497
. : milestone, 476,
iast_INACTIVE (452.448 µs) : 430, 474
. : milestone, 452,
iast_TELEMETRY_OFF (466.463 µs) : 446, 487
. : milestone, 466,
tracing (447.387 µs) : 426, 468
. : milestone, 447,
section candidate
no_agent (368.761 µs) : 348, 389
. : milestone, 369,
iast (476.246 µs) : 455, 497
. : milestone, 476,
iast_FULL (545.913 µs) : 525, 567
. : milestone, 546,
iast_GLOBAL (502.543 µs) : 481, 524
. : milestone, 503,
iast_HARDCODED_SECRET_DISABLED (478.54 µs) : 458, 499
. : milestone, 479,
iast_INACTIVE (454.297 µs) : 432, 476
. : milestone, 454,
iast_TELEMETRY_OFF (471.834 µs) : 450, 493
. : milestone, 472,
tracing (439.729 µs) : 419, 460
. : milestone, 440,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.35.0-SNAPSHOT~e650df895a, baseline=1.37.0-SNAPSHOT~179ce68ab6
dateFormat X
axisFormat %s
section baseline
no_agent (1.464 ms) : 1453, 1476
. : milestone, 1464,
appsec (2.25 ms) : 2213, 2286
. : milestone, 2250,
iast (1.987 ms) : 1943, 2030
. : milestone, 1987,
iast_GLOBAL (2.037 ms) : 1992, 2081
. : milestone, 2037,
profiling (1.879 ms) : 1844, 1914
. : milestone, 1879,
tracing (1.855 ms) : 1821, 1890
. : milestone, 1855,
section candidate
no_agent (1.471 ms) : 1459, 1483
. : milestone, 1471,
appsec (2.209 ms) : 2175, 2243
. : milestone, 2209,
iast (1.965 ms) : 1923, 2006
. : milestone, 1965,
iast_GLOBAL (2.005 ms) : 1963, 2046
. : milestone, 2005,
profiling (1.844 ms) : 1811, 1876
. : milestone, 1844,
tracing (1.841 ms) : 1809, 1874
. : milestone, 1841,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.35.0-SNAPSHOT~e650df895a, baseline=1.37.0-SNAPSHOT~179ce68ab6
dateFormat X
axisFormat %s
section baseline
no_agent (14.717 s) : 14717000, 14717000
. : milestone, 14717000,
appsec (15.106 s) : 15106000, 15106000
. : milestone, 15106000,
iast (18.841 s) : 18841000, 18841000
. : milestone, 18841000,
iast_GLOBAL (17.857 s) : 17857000, 17857000
. : milestone, 17857000,
profiling (15.365 s) : 15365000, 15365000
. : milestone, 15365000,
tracing (14.871 s) : 14871000, 14871000
. : milestone, 14871000,
section candidate
no_agent (15.37 s) : 15370000, 15370000
. : milestone, 15370000,
appsec (15.056 s) : 15056000, 15056000
. : milestone, 15056000,
iast (19.008 s) : 19008000, 19008000
. : milestone, 19008000,
iast_GLOBAL (17.817 s) : 17817000, 17817000
. : milestone, 17817000,
profiling (14.978 s) : 14978000, 14978000
. : milestone, 14978000,
tracing (15.106 s) : 15106000, 15106000
. : milestone, 15106000,
|
b0d0e89
to
8b94822
Compare
Debugger benchmarksParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 7 unstable metrics. See unchanged results
Request duration reports for reportsgantt
title reports - request duration [CI 0.99] : candidate=None, baseline=None
dateFormat X
axisFormat %s
section baseline
noprobe (309.134 µs) : 247, 372
. : milestone, 309,
basic (312.871 µs) : 299, 326
. : milestone, 313,
loop (12.405 ms) : 12374, 12436
. : milestone, 12405,
section candidate
noprobe (297.086 µs) : 261, 333
. : milestone, 297,
basic (314.752 µs) : 305, 325
. : milestone, 315,
loop (12.337 ms) : 12303, 12372
. : milestone, 12337,
|
9bba04a
to
c2f4273
Compare
c2f4273
to
592d31f
Compare
benchmarks are run only when a file is modified on paths under dd-java-agent/agent-debugger/*
592d31f
to
e650df8
Compare
What Does This Do
benchmarks are run only when a file is modified on paths under
dd-java-agent/agent-debugger/*
Motivation
Benchmarks run for every debugger PR to assess improvement/regression/neutral
Additional Notes
Jira ticket: DEBUG-2440